Methods and apparatus for stretched light field microscope

ABSTRACT

The information budget of a light field microscope is increased by increasing the field of view and image circle diameter of the microscope, while keeping the ratio of overall magnification of the microscope to the numerical aperture of the microscope unchanged. Alternatively, the information budget is increased by increasing the field of view and image circle diameter of the microscope by a first factor, while increasing the ratio of overall magnification of the microscope to the numerical aperture of the microscope by a smaller, second factor. In some cases, an infinity-corrected light field microscope has an overall magnification that is greater than the nominal magnification of the objective lens.

RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.14/868,340 filed on Sep. 28, 2015, which claims the benefit of U.S.Provisional Patent Application No. 62/056,585, filed Sep. 28, 2014 (the“Provisional Application”).

FIELD OF TECHNOLOGY

The present invention relates generally to methods and apparatus forincreasing the number of diffraction-limited resolvable spots that alight field microscope (LFM) captures.

SUMMARY

The inventors were faced by a problem: How to increase the informationbudget—that is, the number of diffraction-limited resolvable spots—thata light field microscope (LFM) captures.

In illustrative implementations of this invention, the problem is solvedas follows: The information budget of a LFM is increased by increasingthe field of view and image circle diameter of the LFM, while keepingthe ratio of overall magnification of the LFM to the numerical apertureof the LFM unchanged. Alternatively, the information budget is increasedby increasing the field of view and image circle diameter of the LFM bya first factor, while increasing the ratio of overall magnification ofthe LFM to the numerical aperture of the LFM by a smaller, secondfactor.

In some implementations, the information budget of the LFM is increased,but the numerical aperture and overall magnification of the LFM are keptunchanged.

Increasing the information budget of the LFM has many practicalbenefits. For example, in some cases, increasing the information budgetof an LFM allows the microscope to examine a larger portion of thespecimen in a single image because of a larger field of view (FOV). Thisis highly advantageous for examining dynamic samples (e.g.,neurobiological tissue samples or other biological samples). Also, insome cases, increasing the information budget may increase the axialdepth of an image.

In some embodiments of this invention, the LFM comprises an infiniteconjugate LFM—that is, an LFM with an infinity-corrected objective.

In some embodiments, the information budget of an infinite conjugate LFMis increased by a method that comprises the following three steps: StepOne: Increase the numerical aperture of the LFM by a factor of X, whereX is a real number greater than or equal to 1. (Note that X may be 1, ifany increase—or further increase—in numerical aperture is notpracticable). Step Two: Increase the focal length of the objective by afactor of Y such that the overall magnification of the LFM is reduced inStep Two by a factor of Y, where Y is a real number greater than 1. Thefirst and second steps together increase the information budget of theLFM by (X*Y)² and together reduce the diffraction-limited resolvablespot size (“spot size”) of the LFM by a factor of X*Y. Step Three:increase the focal length and diameter of the tube lens by a factor Z,where Z is a real number that is greater than 1 and that is sufficientlylarge, after giving effect to Steps One, Two and Three, that theresolution of the LFM sensor is not less than the resolution limit ofthe real image incident on the sensor. (If, before any the three stepsare taken, the LFM sensor is at the resolution limit of the real imageincident to it, then Z is equal to or greater than X*Y. Note that theoverall magnification of the LFM is increased by a factor of Z in Step3, so as to at least partially compensate for the reduction in overallmagnification in Step Two.) An overall effect of this three-step methodis that information budget is increased by a factor of (X*Y)². The valueZ does not affect the information budget but the ability of the sensorto resolve the real optical image created by the LFM.

In some cases, the three-step method described in the immediatelypreceding paragraph is supplemented by increasing the diameter of themicroscope tube. In these cases: (a) the tube lens forms an image circleat the back focal plane of the tube lens; (b) the third step in theimmediately preceding paragraph (increasing the focal length of the tubelens) would, in the absence of any obstruction of light, increase thediameter of the image circle by a factor of Z, resulting in an expandedimage circle; (c) for each respective point along a longitudinal axis ofthe tube, the tube has an internal tube diameter that is equal to thediameter of the tube from inner wall to inner wall of the tube at thatrespective point; and (d) the method includes a supplemental step.Specifically, the supplemental step comprises increasing the internaltube diameter, at each point along the longitudinal axis by an amount,if any, at least sufficient to cause the tube to not obstruct any lightfrom the objective lens that would otherwise pass through the tube lensand travel to the expanded image circle. However, this supplemental stepis not necessary in all cases, because, among other things, the internaltube diameter is initially large enough in some cases that no adjustmentto it is needed.

In some embodiments of this invention, to increase the informationbudget of an infinite conjugate LFM, the LFM is “stretched”—that is, thefocal length of the tube lens is increased. Light from each object pointexits the objective as a bundle of parallel light rays, but ray bundlesfrom different object points exit the objective in different directions.The increased tube lens focal length causes the ray bundles to spreadout more from each other before reaching the tube lens, so that thediameter of the circle of light striking the tube lens increases, andthus the diameter of the image formed by the tube lens increases.

The information budget of the LFM is proportional to the image circlediameter (i.e., in this case, diameter of the real image formed by thetube lens) and is inversely proportional to the resolvable spot size.The resolvable spot size is proportional to the overall magnification ofthe LFM and inversely proportional to the numerical aperture of the LFM.

Thus, increasing the image circle diameter (by lengthening the tube lensfocal length, and increasing the tube lens diameter and image circlediameter) while keeping the overall magnification and numerical apertureof the LFV unchanged, causes the information budget of the LFM toincrease.

The overall magnification of an infinite conjugate LFM is equal to thefocal length of the tube lens divided by the focal length of theobjective.

In some cases, it is desirable to keep the overall magnification of aninfinite conjugate LFM unchanged while increasing the informationbudget. In order to keep the overall magnification unchanged, the focallength of the objective is increased by the same factor as the focallength of the tube lens, diameter of the tube lens, and image circlediameter.

In some cases, it is desirable to adjust the overall magnification ornumerical aperture of an infinite conjugate LFM, while increasing theinformation budget. Increasing the image circle diameter by a firstfactor and increasing the ratio of the overall magnification of the LFMto the numerical aperture of the LFM by a smaller, second factorincreases the information budget, in illustrative embodiments of thisinvention. For example, in order to increase the information budget andoverall magnification while keeping numerical aperture unchanged, twosteps are taken in some cases: First, the tube lens' focal length anddiameter is increased appropriately to yield a desired increase in imagecircle diameter. Then the objective lens focal length is adjusted toyield a desired change in magnification, subject to the constraint thatthe objective lens focal length is increased by some positive amount.

The following three examples help to illustrate the above concepts,regarding an infinite conjugate LFM:

(1) If one starts with an infinite conjugate LFM with an objective thathas a nominal magnification of 40× and then doubles the tube lens' focallength and diameter but keeps the numerical aperture of the LFM and theobjective lens focal length unchanged, then both the image circlediameter and the overall magnification will double, but the informationbudget will not change. This is because the resolvable spot size willalso double.

(2) If one starts with an infinite conjugate LFM with an objective thathas a nominal magnification of 40× and then doubles the tube lens' focallength and diameter, reduces the nominal magnification of the objectiveto 20× (i.e., doubles the focal length of the objective) and keepsnumerical aperture unchanged, then (i) the overall magnification will beunchanged, (ii) the image circle diameter will double, and (iii) theinformation budget will quadruple.

(3) If one starts with an infinite conjugate LFM with an objective thathas a nominal magnification of 40×, and then doubles the tube lens'focal length and diameter, reduces the nominal magnification of theobjective to 30× (i.e., increases the focal length of the objective by afactor of 1.333) and keeps the numerical aperture unchanged, then (i)the overall magnification will increase by a factor of 1.333, (ii) theimage circle diameter will increase by a factor of 1.333, and (iii) theinformation budget will increase by a factor of 1.778 (that is, 1.333²).

In some cases, in order to increase the information budget of aninfinite conjugate LFM, four changes are made: (1) the focal length ofthe objective lens is increased by a factor of K (thereby decreasing thenominal magnification of the objective lens by a factor of K); (2) thefocal length of the tube lens is increased by a factor of K; (3) thetube lens' diameter is increased by a factor of K; and (4) as a resultof increasing the tube lens focal length by a factor of K, the distanceof the tube lens from the telecentric stop is increased by a factor of Kand the distance of the tube lens from the image plane is increased by afactor of K as well. As used herein, K is a variable that is a positivereal number. These changes: (a) cause the field of view (FOV) of the LFMto increase by a factor of K; and (b) cause the image circle diameterD_(img) to increase by a factor of K and the information budget toincrease by a factor of K². These changes may be made without alteringthe numerical aperture (NA) of the LFM (e.g., if the objective withlower nominal magnification has the same NA as the objective with highernominal magnification). Furthermore, these changes do not alter theoverall magnification of the infinite conjugate LFM, which is equal tothe focal length of the tube lens divided by the focal length of theobjective lens. Because the focal length of the tube lens is increasedby a factor of K and the focal length of the objective is also increasedby a factor of K, the overall magnification of the infinite conjugateLFM does not change. If NA is kept unchanged, then the overall effect ofthe modification is to increase the information budget by a factor of K²and to keep the overall magnification of the LFM unchanged.

In some embodiments of this invention, the LFM comprises a finiteconjugate LFM—that is, an LFM with a finite conjugate objective.

To increase the information budget of the finite conjugate LFM, the LFMis “stretched” by adding a second magnification stage in the opticalpath—that is, by inserting a reimaging lens in the optical path afterthe intermediate image formed by the objective. The reimaging lensreimages and magnifies the intermediate image that is formed by theobjective. The diameter of the real image formed by the reimaging lensis larger than the diameter of the intermediate image formed by theobjective.

The information budget of the finite conjugate LFM is proportional tothe image circle diameter (i.e., in this case, the diameter of the realimage formed by the reimaging lens) and is inversely proportional to theresolvable spot size. The resolvable spot size is proportional to theoverall magnification of the LFM and inversely proportional to thenumerical aperture. Thus, increasing the image circle diameter (due tomagnification by the reimaging lens) without changing the overallmagnification of the LFM and without changing the numerical aperture ofthe LFM increases the information budget of the LFM.

The overall magnification of the finite conjugate LFM depends, in part,on where the reimaging lens is placed relative to (i) the intermediateimage plane and (ii) the image plane at the lenslet array. According toa thin lens model, the magnification created by the relay lens is equalto a first optical distance divided by a second optical distance, where:(a) the first optical distance is between the relay lens and the lensletarray; and (b) the second optical distance is between the relay lens andthe intermediate image plane.

In some cases, it is desirable to keep the overall magnification of afinite conjugate LFM unchanged while increasing the information budget.In order to keep the overall magnification unchanged, the magnificationof the objective is decreased by the same factor as the magnificationcreated by the reimaging lens. For example, if the reimaging lens aloneproduces a magnification of 3×, then the magnification of the objectiveis decreased by a factor of 3, in order to compensate. The magnificationof the objective is adjusted by adjusting the focal length of theobjective. For example, to decrease the magnification of the objectiveby a factor of 3, the focal length of the objective is increased by afactor of 3.

In other cases, it is desirable to adjust the overall magnification ornumerical aperture of a finite conjugate LFM, while increasing theinformation budget. Increasing the image circle diameter by a firstfactor and increasing the overall magnification by a smaller, secondfactor increases the information budget. Thus, in order to adjust theoverall magnification of a finite conjugate LFM, while also increasingthe information budget, two steps are taken: First, the LFM is stretchedby adding a second magnification stage, in which the reimaging lens ispositioned appropriately to yield a desired magnification and desiredincrease in image circle diameter. Then the magnification of theobjective is decreased in order to yield a desired change in overallmagnification of the LFM, subject to the constraint that themagnification of the objective lens is decreased by some amount.

The following three examples help to illustrate the above concepts,regarding a finite conjugate LFM:

(1) If one starts with a finite conjugate LFM with an objective that hasa nominal magnification of 40× and then (while keeping the objectivefocal length and numerical aperture unchanged) adds a magnificationstage in which the reimaging lens is positioned at 1.5 times its focallength from the intermediate image and at 3 times its focal length fromthe lenslet array, then, according to a thins lens formulation: (i) theoverall magnification of the LFM will double, (ii) the image circlediameter will double, and (iii) the information budget will remainunchanged.

(2) If one starts with a finite conjugate LFM with an objective that hasa nominal magnification of 40× and then, without changing the numericalaperture of the LFM, both (a) reduces the nominal magnification of theobjective to 20× (i.e., doubles the focal length of the objective), and(b) adds a magnification stage in which the reimaging lens is positionedat 1.5 times its focal length from the intermediate image and at 3 timesits focal length from the lenslet array, then: (i) (in a thin lensmodel) the overall magnification will be unchanged, (ii) the imagecircle diameter will double, and (iii) the information budget willquadruple.

(3) If one starts with a finite conjugate LFM with an objective that hasa nominal magnification of 40× and then, without changing the numericalaperture of the LFM, both (a) reduces the nominal magnification of theobjective to 30× (i.e., increases the focal length of the objective by afactor of 1.333), and (b) adds a magnification stage in which thereimaging lens is positioned at 1.5 times its focal length from theintermediate image and at 3 times its focal length from the lensletarray, then (i) (in a thin lens model) the overall magnification willincrease by a factor of 1.333, (ii) the image circle diameter willincrease by a factor of 2, and (iii) the information budget willincrease by a factor of 1.778 (that is, 1.333²).

In some cases, in order to increase the information budget (withoutchanging numerical aperture or overall magnification) of a finiteconjugate LFM, two changes are made: (1) the focal length of theobjective lens is increased by a factor of K (thereby decreasing themagnification of the objective lens by a factor of K); (2) a reimaginglens (i.e., magnifying relay lens) is inserted in the optical path afteran intermediate image plane and before a lenslet array. These changes:(a) cause the field of view of the LFM to increase by a factor of K; and(b) cause the image circle diameter D_(img) to increase by a factor of Kand the information budget to increase by a factor of K². These changesmay be made without altering the numerical aperture (NA) of the LFM(e.g., if the objective with lower nominal magnification has the same NAas the objective with higher nominal magnification). Furthermore, thesechanges do not alter the overall magnification. The reimaging lens ispositioned such that the magnification created by the reimaging lens isequal to K. For example, in a thin lens model, the magnification createdby the relay lens is equal to a first optical distance divided by asecond optical distance, where: (a) the first optical distance isbetween the relay lens and the new position of the lenslet array (thelenslet array being farther from the objective than before the changes);and (b) the second optical distance is between the relay lens and theintermediate image plane (which is positioned where the lenslet arraywas located before the changes). The decrease in magnification of theobjective lens is compensated for by the increase in magnification dueto the relay magnifying lens, so there is no change in overallmagnification. If NA is kept unchanged, the overall effect of themodification is to increase the information budget by a factor of K² andto keep the overall magnification of the LFM unchanged.

In some implementations of this invention, the overall magnification ofthe LFM does not change. However, in other implementations of thisinvention, the overall magnification does change. Thus, in someembodiments, the field of view, image circle diameter and informationbudget of the LFM increase, and the overall magnification also changes.For example, a change in magnification may occur for an infiniteconjugate LFM, if the focal length of the tube lens and the focal lengthof the objective lens are scaled by different amounts. Likewise, achange in magnification may occur for a finite conjugate LFM, if thereimaging lens does not compensate exactly for the decrease in themagnification of the objective lens.

In some implementations, the diameter of at least a portion of themicroscope tube is increased to order to make room for the increasedcircle size diameter. For example, in some cases with a finite conjugateobjective, the diameter of the microscope tube is increased in a regionof the tube that is after the intermediate image formed by theobjective. In some cases with an infinity-corrected objective, thediameter of the entire microscope tube is increased. In some cases, witheither an infinite conjugate LFM or finite conjugate LFM, some regionsof the microscope tube may be portions of cones.

In some embodiments of this invention, the field of view, image circlediameter and information budget of the LFM increase, and one or more ofthe following also changes: overall magnification of the LFM, numericalaperture of the LFM, and diffraction-limited resolvable spot size.

An advantage of the present invention is that it may easily befabricated with a standard, off-the-shelf objective lens.

In some embodiments of this invention: (a) the LFM is an infiniteconjugate LFM; (b) the tube lens focal length is longer than the 200 mm;and (c) the magnification at the image plane is greater than the nominalmagnification of the objective lens.

In some embodiments of this invention: (a) the LFM is a finite conjugateLFM; and (b) a magnifying lens is interposed between the objective lensand the image plane, causing the magnification at the image plane to begreater than the magnification produced by the objective lens.

In illustrative implementations of this invention, the LFM includesmultiple lenslet arrays.

In some cases, a higher lenslet packing density improves the utilizationof the information budget of the LFM (i.e., the number ofdiffraction-limited resolvable spots captured by the LFM).

In some cases, the lenslet packing density is increased by changing thepacking configuration within a single lenslet array. For example, insome cases, lenslets are packed in a hexagonal pattern rather than arectangular pattern, because the former has a higher packing densitythan the latter.

In some cases, staggered lenslet arrays are used to increase theeffective lenslet packing density. The staggered arrays create overlapregions in an image.

In some cases, the staggering (overlapping) of lenslet arrays isachieved by effectively, but not actually, placing lenslet arrays inseries. For example, in some cases, beamsplitters and a mirror arepositioned in series in an optical path of the LFM, at different opticaldistances from the objective of the LFM. Each of these beamsplitters andmirror, respectively, steers light through a lenslet array to a separateimage sensor. Images captured by the image sensors are then combined,

Alternatively, in some cases, the staggering (overlapping) of lensletarrays is achieved by lenslet arrays that are all at the same opticaldistance from the objective. For example, in some cases: (a)beamsplitters and a mirror direct light to multiple lenslet arrays thatare each the same optical distance from the objective lens; (b) aseparate image sensor is used for each lenslet array, such that eachimage sensor, respectively, images one of the lenslet arrays; (c) thelenslet arrays are positioned at different lateral positions relative tothe imaging planes (that is, the distance that the lenslet arrays arelaterally shifted, relative to their respective imaging sensors, variesfrom one array to another array); and (d) when images captured by theimage sensors are combined in post-processing, the effect is similar towhat would occur if all of the lenslet arrays were in front of a singleimage sensor and were staggered (laterally shifted) relative to eachother.

In illustrative implementations, the staggering of lenslet arraysincreases effective packing density and thus improves utilization of theinformation captured by the LFM. For example, in some cases, thestaggering of lenslet arrays improves sampling of data and lateral andaxial resolution of the LFM.

The description of the present invention in the Summary and Abstractsections hereof is just a summary. It is intended only to give a generalintroduction to some illustrative implementations of this invention. Itdoes not describe all of the details and variations of this invention.Likewise, the descriptions of this invention in the Field of Technologysection and Field Of Endeavor section are not limiting; instead theyeach identify, in a general, non-exclusive manner, a technology to whichexemplary implementations of this invention generally relate. Likewise,the Title of this document does not limit the invention in any way;instead the Title is merely a general, non-exclusive way of referring tothis invention. This invention may be implemented in many other ways.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an infinite conjugate light field microscope (LFM).

FIG. 2 shows an example of an infinite conjugate LFM, in which acompound tube lens comprises four doublets.

FIG. 3 shows a finite conjugate LFM.

FIG. 4 shows an example of an LFM, in which image sensors areeffectively positioned in series, such that they capture light atdifferent optical distances from the objective lens.

FIG. 5 shows examples of lenslet packing and of imagelet overlaps due tostaggered lenslet arrays.

FIG. 6 shows a light field imager, including a lenslet array, set offiber optic tapers, and image sensors.

FIG. 7 shows another example of a light field imager.

FIG. 8 shows relay lenses that image the narrow ends of fiber optictapers onto image sensors.

FIG. 9 and FIG. 10 each show an example of prisms steering light fromdifferent regions of a lenslet array to different cameras.

FIG. 11A shows two light field imagers at the same optical distance fromthe objective lens.

FIG. 11B shows three light field imagers at the same optical distancefrom the objective lens.

FIG. 11C shows four light field imagers at the same optical distancefrom the objective lens.

FIG. 11D shows five light field imagers at the same optical distancefrom the objective lens.

FIG. 11E shows a more compact configuration, in which beamsplitters andprisms steer light to five light field imagers at the same opticaldistance from the objective lens.

FIGS. 12 and 13 each show a scanning unit for actuating motion of asensor to scan a light field image. In FIG. 12, the scanning movement isin a straight line. In FIG. 13, the scanning movement is zig-zag.

FIG. 14 shows hardware for controlling an LFM.

The above Figures show some illustrative implementations of thisinvention, or provide information that relates to those implementations.However, this invention may be implemented in many other ways.

DETAILED DESCRIPTION

The information budget or total available information content from alight field microscope (LFM) is limited by the number of resolvablespots, which in turn is limited by diffraction.

Recent super-resolution and deconvolution techniques better utilize theexisting information by exploiting aliasing through multiple views andthe known point spread function (PSF). While super-resolution anddeconvolution better utilize the available information budget, theseapproaches do not increase the total information.

The information budget for a LFM is a function of wave length, numericalaperture, magnification and field of view (FOV). This is true for anordinary (non-light field) microscope also.

For a LFM, there is also a tradeoff between angular and specialresolution of the LFM microscope. For a LFM, the tradeoffs for a givenfixed information budget are as follows. One can: (i) reduce lensletsize to increase the spatial resolution at the expense of angularresolution; (ii) increase lenslet size to increase the angularresolution at the expense of spatial resolution. (iii) increasemagnification to increase both angular and spatial resolution at theexpense of FOV, or (iv) reduce magnification to increase FOV at theexpense of both angular and spatial resolution.

In illustrative implementations, for a given NA, magnification andwavelength of a LFM, the information budget is increased by increasingthe FOV of the LFM.

The diffraction limited resolvable spot size, R_(spt), of an LFM with agiven objective is:

$\begin{matrix}{R_{spt} = {\frac{c\;\lambda}{NA}M}} & \left( {{Equation}\mspace{14mu} 1} \right)\end{matrix}$where c is a constant defining the criterion used (c=0.61 for Rayleighcriterion; c=0.47 for Sparrow criterion), λ is the wavelength of light,NA is the numerical aperture of the objective and M is the overallmagnification of the LFM.

The densest possible lenslet packing for a circularly resolvable spot isachieved through hexagonal packing, which, has a packing density, η_(h),of η_(h)=⅙π√{square root over (3)}≈0.9069. Thus, an upper limit of thenumber of diffraction-limited resolvable spots or “the informationbudget”, i, for an image circle diameter D_(img) and resolvable spotsize R_(spt), is given by:

$\begin{matrix}{i = {{\eta_{h}\frac{{\pi\left( {0.5D_{img}} \right)}^{2}}{{\pi\left( {0.5R_{spt}} \right)}^{2}}} = {\eta_{h}\;\frac{\left( D_{img} \right)^{2}}{\left( R_{spt} \right)^{2}}}}} & \left( {{Equation}\mspace{14mu} 2} \right)\end{matrix}$

For example: Given an image circle diameter equal to a standard 23 mmmicroscope tube diameter, a 40× magnification, 0.95 NA objective, greenlight (535 nm), and applying the Sparrow resolution criterion withhexagonal packing, the information budget is 4,279,952 resolvable spots,roughly the size of a 2068×2068 sensor.

In some cases, a lenslet array with a matched numerical aperture is usedto redistribute the existing information in order to obtain multipleviews from multiple sub-apertures. For example, a 125 μm lenslet arraypitch, with the resolvable spot size of R_(spt)=10.58 for the abovementioned parameters, will provide 99 different angular views (withresolvable spots arranged in a hexagonal lattice). The lenslet arraydoes not change the resolvable spot size R_(spt).

Equation 2 demonstrates that, with a fixed packing density, increasingthe information budget is achievable by either reducing R_(spt) or byincreasing D_(img). Reducing R_(spt) is physically achievable byincreasing the numerical aperture of the objective, or by reducing theoptical magnification. For a given application, however, the numericalaperture and overall magnification may be specified. Thus, it isadvantageous to increase D_(img) in order to increase the informationbudget, even when numerical aperture and overall magnification remainunchanged.

In some embodiments of this invention, the image circle diameter D_(img)of an LFM and the FOV are increased, but the numerical aperture, overallmagnification, and diffraction-limited resolvable spot size of the LFMare unchanged.

In many implementations of this invention, a standard off-the-shelfobjective lens is used, simplifying the creation and replication of thestretched light field microscope.

Infinite Conjugate LFM

FIG. 1 shows an infinite conjugate light field microscope (LFM), in anillustrative implementation of this invention.

In FIG. 1, an infinite conjugate LFM includes an infinity-correctedobjective lens 101, a telecentric stop 103, a tube lens 107, a lensletarray 109 and a set of one or more image sensors 111. The object beingimaged 105 and the telecentric stop 103 are located at the front focalplane and back focal plane, respectively, of the objective 101. Thetelecentric stop 103 and lenslet array 109 are located at the frontfocal plane and back focal plane, respectively, of the tube lens 107.Thus, the telecentric stop 103 is positioned so that it coincides withboth the back focal plane of the objective and the front focal plane ofthe tube lens. This causes the projection at the image plane to betelecentric. The tube lens 107 focuses a real image onto the lensletarray 109.

In FIG. 1, the lenslet array 109 forms imagelets on an image sensor 111.Each imagelet is a “set of views” for a given lenslet in the lensletarray. In some cases, the image sensors 111 are located at an opticaldistance from the lenslet array 109 that is equal to the focal length ofthe lenslet array 109. The LFM is double telecentric (i.e., its entranceand exit pupils are at optical infinity).

In FIG. 1, f1 is the focal length of the objective 101; f2 is the focallength of the tube lens 107, f3 is the focal length of lenslet array109, d1 is the diameter (perpendicular to the optical axis) of tube lens107, and d2 is the tube diameter of the LFM. Typically, tube diameter d2is slightly larger than tube lens diameter d1.

In conventional infinity-corrected LFMs, the objective lens is designedto work with a tube lens that has a focal length that is 200 mm or less.

In contrast, in the example of this invention shown in FIG. 1: (a) thefocal length f2 of the tube lens 107 is greater than 200 mm. Forexample, in some cases, the focal length f2 of tube lens 107 shown inFIG. 1 is at least 225 mm, or at least 250 mm, or at least 275 mm, or atleast 300 mm, or at least 325 mm, or at least 350 mm, or at least 375mm, or at least 400 mm, or at least 425 mm, or at least 450 mm, or atleast 475 mm, or at least 500 mm, or at least 550 mm, or at least 600mm, or at least 1000 mm.

In illustrative implementations, the infinite conjugate LFM istelecentric at both ends. The fact that the LFM is double telecentric(both the entrance and exit pupils of the LFM are at optical infinity)is advantageous for light field imaging with a lenslet array. This isbecause the lenslet array is usually designed for light that strikes thearray at an angle very close to perpendicular to the center plane oflenslet array.

As used herein, “lens” means a single lens or compound lens. In manyimplementations, an infinity-corrected objective and a tube lens areeach a compound lens. Similarly, in many other embodiments, a finiteconjugate objective and a magnifying relay lens are each a compoundlens.

FIG. 2 shows an example of an infinite conjugate LFM, in an illustrativeembodiment of this invention. In FIG. 2, a compound tube lens 107comprises four achromatic doublets 121, 122, 123, 124.

In FIGS. 1 and 2, an infinity-corrected light field microscope has anoverall magnification that is greater than the nominal magnification ofthe objective lens.

In some embodiments, an infinite conjugate microscope does not includeany magnifying optical element other than the tube lens and theobjective lens.

In some embodiments, an infinite conjugate LFM (such as the LFM shown inFIG. 1 or 2) has an information budget that is greater by a factor of K²than a modified information budget. The modified information budget isthe information budget that the LFM would have if the LFM were modifiedby reducing the focal length of the objective lens and the focal lengthand diameter of the tube lens by a factor of K without changing thenumerical aperture and overall magnification of the LFM.

In some alternative embodiments, an infinite conjugate LFM (such as theLFM shown in FIG. 1 or 2) has an information budget that is greater by afactor of K² than the information budget that the microscope would haveif the microscope were modified (i) by reducing the focal length anddiameter of the tube lens by a factor of K and (ii) by increasing theratio of the nominal magnification of the objective lens to thenumerical aperture of the objective lens by a factor of K.

Finite Conjugate LFM

In some embodiments of this invention, a finite conjugate LFM isemployed.

In FIG. 3, a finite conjugate LFM includes a finite conjugate objectivelens 301, telecentric stop 303, magnifying relay lens 307, lenslet array309 and a set of one or more image sensors 311. The LFM in FIG. 3 istelecentric in one direction, because stop 303 is positioned is at theback focal plane of the objective. The object being imaged 305 islocated at a working distance wd from the objective 301. The relay lens107 reimages light from the intermediate image 315 and focuses acircular image (which is a real image) onto the lenslet array 109. Thiscircular image has a diameter d8. In some cases, the relay lens istelecentric on its image side (but not on both sides).

The diameter D_(img) of the image circle is d7 in FIG. 1 and is d8 inFIG. 3.

In FIG. 3, the lenslet array 309 refracts or diffracts light, to formcircular imagelets on a set of one or more image sensors 311. In manycases, the image sensors 311 are located at an optical distance from thelenslet array 309 that is equal to the focal length of the lenslet array309.

In FIG. 3: f9 is the distance between the telecentric stop 303 and theobjective 301; L4 is the distance between the stop 1303 and theintermediate image plane 315; L5 is the distance between theintermediate image plane 315 and the magnifying relaying lens 307; L6 isthe distance between the magnifying relay lens 307 and the lenslet array309; f7 is the focal length of lenslet array 309; d3 is the diameter(perpendicular to the optical axis) of relay lens 307; and d4 is thetube diameter of the LFM. Typically, tube diameter d4 is slightly largerthan relay lens diameter d3.

In FIG. 3, a finite conjugate light field microscope has an overallmagnification that is greater than the magnification of the objectivelens.

In some alternative embodiments, a finite conjugate LFM (such as the LFMshown in FIG. 3) has an information budget that is greater by a factorof K² than the information budget that the microscope would have if themicroscope were modified (i) removing the reimaging lens to reduce theoverall magnification of the microscope by a factor of K and (ii) byincreasing the ratio of the magnification of the objective lens to thenumerical aperture of the objective lens by a factor of K.

Lenslet Overlaps

In some embodiments, image sensors are effectively positioned in seriesin an optical path, by use of beamsplitters and mirrors.

FIG. 4 shows an example of an LFM, in which image sensors areeffectively positioned in series, such that they capture light atdifferent optical distances from the objective lens, in an illustrativeimplementation of this invention. In FIG. 4, beamsplitter 401,beamsplitter 402 and mirror 403 are arranged in series, such that somelight from the tube lens 107 travels first through beamsplitter 401,then through beamsplitter 402, and then strikes mirror 403. In addition,beamsplitter 401, beamsplitter 402 and mirror 403 each reflect light,such that (a) beamsplitter 401 reflects part of the light from the tubelens to image sensor 411 via lenslet array 421, (b) beamsplitter 402reflects part of the light from the tube lens to image sensor 412 vialenslet array 422, and (c) mirror 403 reflects part of the light fromthe tube lens to image sensor 413 via lenslet array 423. Image sensors411, 412, 413 are housed in camera 420.

Each lens of a lenslet array captures slightly different angularinformation, enabling computational reconstruction of 3-D images fromsingle camera shots for a given sensor. Effectively positioning lensletarrays (and their respective image sensors) in series, such as in theexample shown in FIG. 4, has at least two advantages: it (a) increasesaxial depth acquisition because each have different object-side focalplane, and (b) increases resolution by shifting one lenslet array withrespect to the other (staggering). This series approach is advantageousin overcoming the low resolution “singularity” obtained at each lensletarray a specific focal plane, as long as the axial depth of the lensletarrays overlap. Syncing numerous lenslet/camera pairs increases thesampling rate of the LFM. At least partial redundancy of depthmeasurement is achieved when lenslets axial depth overlap. As lightpasses through a series of beamsplitters, the amount of illuminationdecreases. In some cases, to mitigate this reduction of light intensity,light amplifiers are included in the later optical paths.

In illustrative implementations of this invention, each lenslet arraycomprises an array of microlenses.

In some cases, the utilization of the existing information budget of theLFM (i.e., the number of diffraction-limited resolvable spots capturedby the LFM) is improved by increasing the lenslet packing density.

FIG. 5 shows examples of lenslet packing and of imagelet overlaps due tostaggered lenslet arrays, in an illustrative implementation of thisinvention. In FIG. 5: (a) the top row shows rectangular packing oflenslets; (b) the bottom row shows hexagonal packing of lenslets; (c)the first column shows examples in which there is a single lenslet arrayand no overlap of lenslets; and (d) the second, third, fourth and fifthcolumns show examples in which overlap regions where images formed bylenslet arrays overlap. For purposes of this description of FIG. 5,columns are numbered from left to right, so that the leftmost column iscolumn 1 and the rightmost column in column 5. Specifically, FIG. 5shows: (a) rectangular lenslet packing with no overlap 501; (b)hexagonal lenslet packing with no overlap 502; (c) rectangular lensletpacking, in which images formed by two lenslet arrays overlap 511; (d)hexagonal lenslet packing, in which images formed by two lenslet arraysoverlap 512; (e) hexagonal lenslet packing, in which images formed bythree lenslet arrays overlap 522; (c) rectangular lenslet packing, inwhich images formed by four lenslet arrays overlap 531; (d) hexagonallenslet packing, in which images formed by four lenslet arrays overlap532; and (e) rectangular lenslet packing, in which images formed by fivelenslet arrays overlap 541.

In FIG. 5, each lenslet creates an imagelet at the image plane. Thus,overlapping lenslets create overlapping imagelets. The greater theoverlap (i.e., the greater the number of imagelets that overlap in agiven overlap region), the greater the sampling density. The packingdensity may be expressed in terms of the pitch (distance) between thecenters of the imagelets created by the lenslets. The pitch, in turn,may be expressed in terms of r (the radius of the imagelet) or d (thediameter of the imagelet).

In FIG. 5, the centers of the imagelets are marked by dots, to make iteasier to see the distances between the centers. However, these dotstypically do not exist in the actual imagelets.

The following is a list of packing densities, for the lenslet packingexamples shown in FIG. 5: (a) for lenslet packing 501, the shortestdistance between the centers of the imagelets is d and the longestdistance between the centers of the imagelets is d√2; (b) for lensletpacking 502, the shortest and longest distance between the centers ofthe imagelets is d; (c) for lenslet packing 511, the shortest distancebetween the centers of the imagelets is

$d\;\frac{\sqrt{2}}{2}$and the longest distance between the centers of the imagelets is d; (d)for lenslet packing 512, the shortest distance between the centers ofthe imagelets is

$d\;\frac{\sqrt{3}}{3}$and the longest distance between the centers of the imagelets is d; (e)for lenslet packing 522, the shortest and longest distances between thecenters of the imagelets are equal to r; (f) for lenslet packing 531,the shortest distance between the centers of the imagelets is r and thelongest distance between the centers of the imagelets is r√{square rootover (2)}; (g) for lenslet packing 532, the shortest distance betweenthe centers of the imagelets is

$r\;\frac{\sqrt{3}}{3}$and the longest distance between the centers of the imagelets is r; and(h) for lenslet packing 541, the shortest distance between the centersof the imagelets is

${r\;\frac{\sqrt{2}}{2}},$and the longest distance between the centers of the imagelets is r.Imaging Configurations

In some cases, fiber optic tapers are used to steer light from lensletarray(s) to image sensors. FIG. 6 shows a light field imager, includinga lenslet array, set of fiber optic tapers, and image sensors, in anillustrative implementation of this invention. In the example shown inFIG. 6, light passes through a lenslet array 601, then passes through aset of fiber optic tapers 610 and then strikes a set of image sensors620. Each fiber optic taper, respectively, steers light to a singleimage sensor. For example, tapers 611, 612 and 614 steer light to imagesensors 621, 622 and 624, respectively.

FIG. 7 shows another example of a light field imager, in an illustrativeimplementation of this invention. In FIG. 7, a light 701 strikes lensletarray 702. Light that exits the lenslet array 702 forms a light-fieldimage on the wide face 703 of a fiber optic taper array 704. Each fiberoptic taper in taper array 704 steers light to a single image sensor.For example, tapers 711, 712 and 713 steer light to image sensors 721,722 and 723, respectively. Image sensors 721, 722 and 723 are part ofcameras 731, 732 and 733, respectively. Light field imager 100 comprisesa lenslet array 702, fiber optic taper array 704 and multiple cameras731, 732, 733.

FIG. 8 shows relay lenses that image the narrow ends of fiber optictapers onto image sensors, in an illustrative implementation of thisinvention. In FIG. 8: (a) relay lens 741 relays light from taper 711 toimage sensor 721; (b) relay lens 742 relays light from taper 712 toimage sensor 722; and (c) relay lens 743 relays light from taper 713 toimage sensor 723.

In some embodiments of this invention, prisms are used. For example, insome cases: (a) multiple cameras capture images; (b) the cameras are toolarge for all of the cameras to fit directly behind a lenslet array; and(c) to solve this problem, the cameras are positioned further apart fromeach other and prisms are used to steer light from different parts ofthe lenslet array to different cameras.

In some embodiments: (a) multiple lenslet arrays are positioned at thesame optical distance from the objective; and (b) prisms are used tosteer light to the different lenslet arrays.

FIG. 9 and FIG. 10 each show an example of prisms steering light fromdifferent regions of a lenslet array to different cameras.

In FIG. 9: (a) light travels from taper 711, then through prism 751,then through relay lens 741 to image sensor 721; and (b) light travelsfrom taper 713, then through prism 752, and then through relay lens 743to image sensor 723.

In FIG. 10, light travels through imagelets plane 1001. Then differentportions of the light travel through different fiber optic tapers intaper array 1003. Light exiting some of tapers (e.g., 1009) travelsdirectly to a camera (e.g., 1012) without passing through a prism. Lightexiting some other tapers passes through a prism (e.g., 1005, 1006,1007, 1008) that steers the light to a camera (e.g., 1010, 1011, 1012,1013, 1014, 1015).

FIG. 11A shows two light field imagers at the same optical distance fromthe objective lens, in an illustrative implementation of this invention.In FIG. 11A, a double imager unit 200 comprises two light field imagers100 and a beamsplitter unit 201. Beamsplitter unit 201 includes abeamsplitter 1110 that splits light 701, such that a portion (e.g.,half) of light 701 is steered to a first light field imager 100 andanother portion (e.g., half) of light 701 is steered to a second lightfield imager 100. Both light field imagers 100 are at the same opticaldistance from the objective.

FIG. 11B shows three light field imagers at the same optical distancefrom the objective lens, in an illustrative implementation of thisinvention. In FIG. 11B, a triple imager unit 300 comprises a doubleimager unit 200, an additional light field imager 100, an additionalbeamsplitter unit 201 and a spacer unit 1101. A spacer unit 1101 isinterposed in the optical path to the light field imager 100 in thebottom of FIG. 11B. The length of spacer unit 1101 is such that allthree light field imagers 100 are at the same optical distance from theobjective.

FIG. 11C shows four light field imagers at the same optical distancefrom the objective lens, in an illustrative implementation of thisinvention. In FIG. 11C, a quadruple imager unit 400 comprises two doubleimager units 200 and an additional beamsplitter 201. All four lightfield imagers are at the same optical distance from the objective.

FIG. 11D shows five light field imagers at the same optical distancefrom the objective lens, in an illustrative implementation of thisinvention In FIG. 11D, a quintuple imager unit comprises a quadrupleimager unit 400, an additional light field imager 100, an additionalbeamsplitter 201, and two spacer units 1101. The length of the twospacer units 1101 is such that all five light field imagers are at thesame optical distance from the objective.

FIG. 11E shows a more compact configuration, in which beamsplitters andprisms steer light to five light field imagers at the same opticaldistance from the objective lens, in an illustrative implementation ofthis invention. In FIG. 11E, a quintuple imager unit comprises twodouble imager units 200, an additional light field imager 100, twoadditional beamsplitter units 201, and two prisms 502. The geometry ofthe beamsplitter units 201 and prisms 502 is such that all five lightfield imagers are at the same optical distance from the objective.

In FIGS. 6-9 and 11A-11E, light 701 has already passed through theobjective lens and one or more other optical elements of the LFM. Thus,loosely speaking, it is light from the objective end of the LFM.

In many cases, one or more light-steering optical elements (e.g., fiberoptic tapers, prisms, mirrors or beamsplitters) are used to direct lightto multiple image sensors such that all of light in the image circle ofthe LFM is captured simultaneously. In these cases, the image sensorsare stationary with respect to the microscope tube of the LFM.

Alternatively, in some cases, an image sensor is scanned over the imagecircle of the LFM.

FIGS. 12 and 13 each show a scanning unit for actuating motion of asensor to scan a light field image, in illustrative embodiments of thisinvention. In FIG. 12, the scanning movement is in a straight line. InFIG. 13, the scanning movement is zig-zag.

In FIG. 12, scanning unit 1200 comprises a linear actuator 1201 andelongated image sensor 1202. Actuator 1201 includes a motor 1205 and alinear track 1207. The actuator moves image sensor 1202 in direction1203 along track 1207, such that the sensor 1202 captures a digitalimage of optical light field image 1208.

In FIG. 13, scanning unit 1300 comprises: (a) a first linear actuator1310 that actuates motion of image sensor 1301 in directions parallel tothe y-axis; and (b) a second linear actuator 1320 that actuates motionof the image sensor 1301 in directions parallel to the x-axis. Thecombined motions actuated by the first and second actuators 1310, 1320cause the image sensor to move in zig-zag path 1303 (or in any otherpath in the x-y plane, including any raster path) such that the sensor1301 captures a digital image of optical light field image 1308.

In FIG. 13, the first linear actuator 1310 comprises a motor 1311 thatactuates motion of the sensor 1301 along track 1312. The second linearactuator 1320 comprises a motor 1321 that actuates motion of the sensor1301 along track 1322.

FIG. 14 shows hardware for controlling an LFM, in an illustrativeimplementation of this invention. One or more cameras 1401 each includeone or more image sensors 1403 for capturing digital images of a lightfield.

Optionally, camera 1401 includes a mechanical scanning unit 1402 formoving an image sensor such that the sensor scans an optical light fieldimage. For example, in some cases, mechanical scanning unit 1402comprises scanning unit 1200 in FIG. 12 or comprises scanning unit 1300in FIG. 13.

A camera controller 1404 controls camera sensor(s) 1403 and, in somecases, also controls the mechanical scanning unit 1402. A data recorder1406 records data captured by the sensor(s) 1403. Data recorder 1406includes a computer (e.g., a microcontroller) 1408 and a memory device1407. An operation controller 1405 controls camera controller 1404 andinterfaces with, and in some cases helps to control, data recorder 1406.In addition, computer 1409 performs image processing computations andoverall control of the LFM. Computer 1409 stores data in memory 1410.

Prototype

The following three paragraphs describe a prototype of this invention.This prototype is a non-limiting example of this invention.

In this prototype, the LFM comprises an infinite conjugate LFM, with aninfinity corrected objective. The tube lens comprises four large (120mm) achromatic doublets (EO-70163). These four lenses comprise twosymmetric pairs to minimize odd order aberrations. The resulting imagearea is 113 cm² which is 27.2 times larger than that created by astandard 23 mm microscope tube lens and 13.2 times larger than thatcreated by a standard 33 mm tube lens. This prototype LFM images avolume of φ2400 μm×600 μm at 50× magnification.

In this prototype with achromatic tube lens, the zero displacementrelative to the chromatic shift is centered in the green wavelengthrange. Filtering the microscope to this green wavelength range offersthe least amount of chromatic dispersion, optimally maintaining thediffraction limit. In order to approximately optimize to this greenwavelength range, an incoherent Halogen illumination is diffused andfiltered to 500-600 nm.

In this prototype, a long working distance (WD) minimizes potentialoptical aberration that is displaced relative to the WD. Specifically,in this prototype, the objective lens comprises a 20× Mitutoyo™ 0.28 NAlong working distance (WD) objective. A long WD is advantageous,because, in many cases, LFMs focus at axial planes that are displacedrelative to their nominal WD.

This invention is not limited to the prototype described in thepreceding three paragraphs; and may be implemented in many differentways.

Field of Endeavor and Problem Faced by the Inventors

A field of endeavor of this invention is increasing the number ofdiffraction-limited resolvable spots that a light field microscope (LFM)captures.

The inventors were faced by a problem: How to increase the informationbudget—that is, the number of diffraction-limited resolvable spots—thata light field microscope (LFM) captures.

In illustrative implementations of this invention, the problem is solvedas follows: The information budget of an LFM is increased by increasingthe field of view and image circle diameter of the microscope, whilekeeping the ratio of overall magnification of the LFM to the numericalaperture of the LFM unchanged. Alternatively, the information budget isincreased by increasing the field of view and image circle diameter ofthe microscope by a first factor, while increasing the ratio of overallmagnification of the LFM to the numerical aperture of the LFM by asmaller, second factor.

Computers

In exemplary implementations of this invention, one or more electroniccomputers (e.g. 1404, 1405, 1408, 1409) are programmed and speciallyadapted: (1) to control the operation of, or interface with, hardwarecomponents of a LFM, including image sensors and any mechanical scanningunit; (2) to perform any digital image processing, digital imageanalysis or computer vision algorithm; (3) to perform any othercalculation, computation, program, algorithm, computer function orcomputer task described or implied above; (4) to receive signalsindicative of human input; (5) to output signals for controllingtransducers for outputting information in human perceivable format; and(6) to process data, to perform computations, to execute any algorithmor software, and to control the read or write of data to and from memorydevices. The one or more computers may be in any position or positionswithin or outside of the LFM. For example, in some cases (a) at leastone computer is housed in or together with other components of the LFM,such as an imaging sensor, and (b) at least one computer is remote fromother components of the LFM. The one or more computers are connected toeach other or to other components in the LFM either: (a) wirelessly, (b)by wired connection, (c) by fiber-optic link, or (d) by a combination ofwired, wireless or fiber optic links.

In exemplary implementations, one or more computers are programmed toperform any and all calculations, computations, programs, algorithms,computer functions and computer tasks described or implied above. Forexample, in some cases: (a) a machine-accessible medium has instructionsencoded thereon that specify steps in a software program; and (b) thecomputer accesses the instructions encoded on the machine-accessiblemedium, in order to determine steps to execute in the program. Inexemplary implementations, the machine-accessible medium comprises atangible non-transitory medium. In some cases, the machine-accessiblemedium comprises (a) a memory unit or (b) an auxiliary memory storagedevice. For example, in some cases, a control unit in a computer fetchesthe instructions from memory.

In illustrative implementations, one or more computers execute programsaccording to instructions encoded in one or more tangible,non-transitory, computer-readable media. For example, in some cases,these instructions comprise instructions for a computer to perform anycalculation, computation, program, algorithm, computer function orcomputer task described or implied above. For example, in some cases,instructions encoded in a tangible, non-transitory, computer-accessiblemedium comprise instructions for a computer to: (1) to control theoperation of, or interface with, hardware components of a LFM, includingimage sensors and any mechanical scanning unit; (2) to perform anydigital image processing, digital image analysis or computer visionalgorithm; (3) to perform any other calculation, computation, program,algorithm, computer function or computer task described or impliedabove; (4) to receive signals indicative of human input; (5) to outputsignals for controlling transducers for outputting information in humanperceivable format; and (6) to process data, to perform computations, toexecute any algorithm or software, and to control the read or write ofdata to and from memory devices.

Actuators

In some implementations, one or more actuators actuate scanning motion,in which one or more imaging sensors move relative to an optical imagebeing scanned. Each of these actuators comprises any kind of actuator,including a linear, rotary, electrical, piezoelectric, electro-activepolymer, mechanical or electro-mechanical actuator. In some cases, theactuator includes and is powered by an electrical motor, including anystepper motor or servomotor. In some cases, the actuator includes a gearassembly, drive train, pivot, joint, rod, arm, or other component fortransmitting motion. In some cases, one or more sensors are used todetect position, displacement or other data for feedback to one of moreof the actuators.

Definitions

The terms “a” and “an”, when modifying a noun, do not imply that onlyone of the noun exists.

To say that a process is “in accordance with” instructions means thatsignals encoding or derived from the instructions are used to controlhardware performing the process. To say that a process is “in accordancewith” instructions does not mean that actual performance must exactlymatch specifications in the instructions. For example, hardwareoperating within tolerances may perform in a manner that does notexactly match the specifications.

To compute “based on” specified data means to perform a computation thattakes the specified data as an input.

Here are some non-limiting examples of a “camera”: (a) a digital camera;(b) a video camera; (c) a light sensor, (d) a set or array of lightsensors; (e) an imaging system; (f) a light field camera or plenopticcamera; (g) a time-of-flight camera; or (h) an optical instrument thatrecords images. A camera includes any computers or circuits that processdata captured by the camera.

The term “comprise” (and grammatical variations thereof) shall beconstrued as if followed by “without limitation”. If A comprises B, thenA includes B and may include other things.

The term “computer” includes any computational device that performslogical and arithmetic operations. For example, in some cases, a“computer” comprises an electronic computational device, such as anintegrated circuit, a microprocessor, a mobile computing device, alaptop computer, a tablet computer, a personal computer, or a mainframecomputer. In some cases, a “computer” comprises: (a) a centralprocessing unit, (b) an ALU (arithmetic logic unit), (c) a memory unit,and (d) a control unit that controls actions of other components of thecomputer so that encoded steps of a program are executed in a sequence.In some cases, a “computer” also includes peripheral units including anauxiliary memory storage device (e.g., a disk drive or flash memory), orincludes signal processing circuitry. However, a human is not a“computer”, as that term is used herein.

The “diameter” of a lens means the clear aperture of the lens.

“Defined Term” means a term or phrase that is set forth in quotationmarks in this Definitions section.

For an event to occur “during” a time period, it is not necessary thatthe event occur throughout the entire time period. For example, an eventthat occurs during only a portion of a given time period occurs “during”the given time period.

The term “e.g.” means for example.

The fact that an “example” or multiple examples of something are givendoes not imply that they are the only instances of that thing. Anexample (or a group of examples) is merely a non-exhaustive andnon-limiting illustration.

As used herein, “field of view” or “FOV” means, for a given image circlein a microscope, the diameter of the image circle divided by the overallmagnification of the microscope. In an infinite conjugate microscope,the given image circle is the image circle of the real image formed bythe tube lens. In the finite conjugate microscope shown in FIG. 3, thegiven image circle is the image circle at the real image formed by thereimaging lens 307.

To reduce (or decrease) by a factor of X means to divide by X. Toincrease by a factor of X means to multiply by X. To increase by afactor of 1 means to leave unchanged. To reduce by a factor of 1 meansto leave unchanged. To increase by a factor of Y, where Y is a positivenumber less than one, means to multiple by Y and thus to reduce. Toreduce by a factor of Y, where Y is a positive number less than one,means to divide by Y and thus to increase.

A “finite conjugate” microscope means a microscope with a finiteconjugate objective lens.

Unless the context clearly indicates otherwise: (1) a phrase thatincludes “a first” thing and “a second” thing does not imply an order ofthe two things (or that there are only two of the things); and (2) sucha phrase is simply a way of identifying the two things, respectively, sothat they each may be referred to later with specificity (e.g., byreferring to “the first” thing and “the second” thing later). Forexample, unless the context clearly indicates otherwise, if an equationhas a first term and a second term, then the equation may (or may not)have more than two terms, and the first term may occur before or afterthe second term in the equation. A phrase that includes a “third” thing,a “fourth” thing and so on shall be construed in like manner.

“For instance” means for example.

Light is “from” an object if the light has at any time reflected from orbeen transmitted through the object or emitted by the object, regardlessof whether the light has subsequently reflected from or been transmittedthrough any other object. For example, light that has passed through anobjective lens and then through a tube lens is “from” both the objectivelens and the tube lens.

In the context of a microscope, “front” is optically closer to theobject being imaged, and “rear” is optically farther from the object,during normal operation of the microscope. In the context of a camera,“front” is optically closer to the scene being imaged, and “rear” isoptically farther from the scene, during normal operation of the camera.In the context of a microscope or camera, the terms “before”, “after”and “behind” shall be construed in like manner.

“Herein” means in this document, including text, specification, claims,abstract, and drawings.

To say that a set of lenslets is “hexagonally packed” means that thelenslets in the set are co-centered with hexagons of a hexagonallattice. The lenslets may be abutting, or may have gaps between them.

As used herein: (1) “implementation” means an implementation of thisinvention; (2) “embodiment” means an embodiment of this invention; (3)“case” means an implementation of this invention; and (4) “use scenario”means a use scenario of this invention.

The term “include” (and grammatical variations thereof) shall beconstrued as if followed by “without limitation”.

An “infinite conjugate” microscope means a microscope that has aninfinity-corrected objective lens.

“Information budget” of a microscope means the maximum number ofdiffraction-limited resolvable spots that can exist at any wavelength oflight in any real image formed by the microscope without modifying themicroscope in any way.

“Intensity” means any measure of or related to intensity, energy orpower. For example, the “intensity” of light includes any of thefollowing measures: irradiance, spectral irradiance, radiant energy,radiant flux, spectral power, radiant intensity, spectral intensity,radiance, spectral radiance, radiant exitance, radiant emittance,spectral radiant exitance, spectral radiant emittance, radiosity,radiant exposure or radiant energy density.

“K” means a variable that is a positive real number. To increase by afactor of K means to multiply by K.

“Lens” means a single lens or a compound lens. Diffractive lens?

As used herein, “lenslet” means a lens. The term “lenslet” does notimply a particular size or range of sizes of a lens.

“Light” means electromagnetic radiation of any frequency. For example,“light” includes, among other things, visible light and infrared light.Likewise, any term that directly or indirectly relates to light (e.g.,“imaging”) shall be construed broadly as applying to electromagneticradiation of any frequency.

“Light field image” means a digital image that contains angle-dependentinformation regarding light incident on a geometric surface, such thatfor each respective angle, out of a set of multiple angles of lightincident at a given position in the surface, the digital image containsinformation regarding intensity of light incident at the given positionfrom the respective angle.

“Light field microscope” or “LFM” means a microscope configured tocapture a light field image.

As used herein, “nominal magnification” of an objective lens of aninfinity-corrected LFM means 200 mm/f, where f is the focal length ofthe objective lens.

“Optical distance” means the distance OD specified in the following twosentences. In a medium of constant refractive index, OD=nd, where n isthe refractive index and d is the geometric length of the light path. Ina medium of varying refractive index, OD=∫_(C) n(s)ds, where C is thelight path, s is distance along light path C, and n is local refractiveindex as a function of distance s. A light path between two points maybe bent (e.g., folded), in which case the geometric length of the lightpath is longer than the straight line physical distance between the twopoints.

The term “or” is inclusive, not exclusive. For example A or B is true ifA is true, or B is true, or both A or B are true. Also, for example, acalculation of A or B means a calculation of A, or a calculation of B,or a calculation of A and B.

As used herein, “parameter” means a variable. For example: (a) ify=f(x), then both x and y are parameters; and (b) if z=f(x(t), y(t)),then t, x, y and z are parameters. A parameter may represent a physicalquantity, such as pressure, temperature, or delay time.

A parenthesis is simply to make text easier to read, by indicating agrouping of words. A parenthesis does not mean that the parentheticalmaterial is optional or may be ignored.

The “pitch” between two lenslets means the distance between (i) thecenter of one of the lenslets and (ii) the center of the other lenslet.

To say that a set of lenslets is “rectangular-packed” means that thelenslets in the set are co-centered with rectangles of a rectangularlattice. The lenslets may be abutting, or may have gaps between them.

To say that a microscope's sensor has a resolution that is “at theresolution limit” of an optical image means that T(F)=0.09 under theRayleigh criterion for at least one wavelength of light, where T is themodulation transfer function of the sensor, F is a spatial frequencylimit of the image, which is equal to 1/S, and S is thediffraction-limited resolvable distance of the microscope as defined byEquation 1 above. To say that a microscope's sensor has a resolutionthat is “not less than the resolution limit” of an optical image meansthat T(F)≥0.09 under the Rayleigh criterion for at least one wavelengthof light, where T is the modulation transfer function of the sensor, Fis a spatial frequency limit of the image, which is equal to 1/S, and Sis the diffraction-limited resolvable distance of the microscope asdefined by Equation 1 above.

As used herein, the term “set” does not include a group with noelements. Mentioning a first set and a second set does not, in and ofitself, create any implication regarding whether or not the first andsecond sets overlap (that is, intersect).

As used herein, “shifted by a distance” means positioned at a distance.To say that X is “shifted by a distance” has no implication regardingwhether or not X is currently moving.

“Some” means one or more.

As used herein, “spot size” of a microscope means the product of (1) thesmallest resolvable distance between two objects as seen on the imageplane of the microscope and (2) the overall magnification of themicroscope.

As used herein, a “subset” of a set consists of less than all of theelements of the set.

“Substantially” means at least ten percent. For example: (a) 112 issubstantially larger than 100; and (b) 108 is not substantially largerthan 100.

The term “such as” means for example.

To say that a machine-readable medium is “transitory” means that themedium is a transitory signal, such as an electromagnetic wave.

Except to the extent that the context clearly requires otherwise, ifsteps in a method are described herein, then the method includesvariations in which: (1) steps in the method occur in any order orsequence, including any order or sequence different than that described;(2) any step or steps in the method occurs more than once; (3) differentsteps, out of the steps in the method, occur a different number of timesduring the method, (4) any combination of steps in the method is done inparallel or serially; (5) any step or steps in the method is performediteratively; (6) a given step in the method is applied to the same thingeach time that the given step occurs or is applied to different thingseach time that the given step occurs; or (7) the method includes othersteps, in addition to the steps described.

This Definitions section shall, in all cases, control over and overrideany other definition of the Defined Terms. For example, the definitionsof Defined Terms set forth in this Definitions section override commonusage or any external dictionary. If a given term is explicitly orimplicitly defined in this document, then that definition shall becontrolling, and shall override any definition of the given term arisingfrom any source (e.g., a dictionary or common usage) that is external tothis document. If this document provides clarification regarding themeaning of a particular term, then that clarification shall, to theextent applicable, override any definition of the given term arisingfrom any source (e.g., a dictionary or common usage) that is external tothis document. To the extent that any term or phrase is defined orclarified herein, such definition or clarification applies to anygrammatical variation of such term or phrase, taking into account thedifference in grammatical form. For example, the grammatical variationsinclude noun, verb, participle, adjective, and possessive forms, anddifferent declensions, and different tenses. In each case described inthis paragraph, the Applicant or Applicants are acting as his, her, itsor their own lexicographer.

Variations

This invention may be implemented in many different ways. Here are somenon-limiting examples:

In some implementations of this invention, the LFM includes a highresolution camera such as a 12 megapixel CMOS (complementarymetal-oxide-semiconductor) camera. Using a high resolution camera in theLFM has numerous advantages, including higher frame rate, acquisition ofmultiple plane, higher resolution, and decrease in stitchingrequirements.

In some implementations, the tube lens comprises apochromatic lenses.Using aprochromatic lenses in the LFM has advantages, including: (i)facilitating image capture over multi-spectral ranges; and (ii) theability to work in multiple fluorescing wavelengths without chromaticaberration.

In some implementations of this invention, wavefront correction improvesthe LFM's resolution. For example, in some cases, the LFM includes aShack-Hartmann wavefront sensor and adaptive optics that correctwavefront distortions due to optical aberrations, including (i) opticalaberrations in lenslet arrays, (ii) optical aberrations due toimproperly positioned lenses, and (iii) optical aberrations createdwithin samples. In some cases, the wavefront correction is dynamic.

In some implementations of this invention, a computer performsalgorithms to computationally correct aberrations. Software correctionsof aberrations in the LFM has numerous advantages, includingfacilitating going below the diffraction limit. For example, in somecases, the computational correction of aberrations includes one or moreof the following: (a) a reconstruction algorithm similar to limitedbaseline computed tomography, but that also accounts for diffraction,and that incorporates an additional deconvolution step for enhancedresolution; or (b) super resolution and deconvolution algorithms toimprove the lateral resolution to below 1 micron.

In illustrative implementations, modifications to a LFM may be made inmany ways, including by (a) physically replacing existing componentswith new components; or (b) by physically modifying existing components.For example, the focal length of a lens may be changed (i) by replacingan existing lens with a new lens; or (ii) by modifying an existing lens(e.g., by grinding it to change its curvature).

In one aspect, this invention is a method of increasing the informationbudget of a microscope, which microscope includes an objective lens, atube lens, and a digital image sensor, the method comprising: (a)increasing the numerical aperture of the microscope by a factor of X,where X is a real number that is not less than 1; (b) increasing thefocal length of the objective lens by a factor of Y, where Y is a realnumber greater than 1; and (c) increasing the focal length and thediameter of the tube lens by a factor of Z, where Z is a real numberthat is greater than 1 and that is sufficiently large such that, aftergiving effect to steps (a), (b) and (c) of this claim 1, the resolutionof the sensor is not less than the resolution limit of the real opticalimage formed by light that is incident on the sensor and that is fromthe objective and tube lenses; wherein: (i) the objective lens isinfinity-corrected; (ii) the microscope is a light field microscope; and(iii) I2 is greater than I₁ by a factor of (X*Y)² where I₁ is theinformation budget of the microscope before steps (a), (b) and (c) ofclaim 1 and I₂ is the information budget of the microscope after steps(a), (b) and (c) of claim 1. In some cases, Y equals Z. In some cases, Yis greater than Z In some cases, M₂=M₁, where M₁ is the overallmagnification of the microscope before steps (a) and (b) in the firstsentence of this paragraph and M₂ is the overall magnification of themicroscope after said steps (a) and (b). In some cases, N₂=N₁, where N₁is the numerical aperture of the microscope before said steps (a) and(b) and N₂ is the numerical aperture of the microscope after said steps(a) and (b). In some cases: (a) the focal length of the tube lens isincreased by physically replacing the tube lens with another tube lens;and (b) the focal length of the objective lens is increased byphysically replacing the objective lens with another objective lens. Insome cases: (a) the tube lens forms an image circle at the back focalplane of the tube lens; (b) step (c) of the first sentence of thisparagraph would, in the absence of any obstruction of light, increasethe diameter of the image circle by a factor of Z, resulting in anexpanded image circle; (c) the microscope includes a tube; (d) lightfrom the objective lens travels inside the tube; (e) for each respectivepoint along a longitudinal axis of the tube, the tube has an internaltube diameter that is equal to the diameter of the tube from inner wallto inner wall of the tube at that respective point; and (f) the internaltube diameter is increased, at each point along the longitudinal axis,by an amount at least sufficient to cause the tube to not obstruct anylight from the objective lens that would otherwise pass through the tubelens and travel to the expanded image circle. Each of the casesdescribed above in this paragraph is an example of the method describedin the first sentence of this paragraph, and is also an example of anembodiment of this invention that may be combined with other embodimentsof this invention.

In another aspect, this invention is a microscope comprising: (a) aninfinity-corrected objective lens; (b) a tube lens for refracting lightfrom the objective lens; (c) one or more lenslet arrays; and (d) one ormore digital image sensors, which image sensors are configured forcapturing light field images; wherein (i) each respective lenslet arrayis positioned such that light from the tube lens passes through therespective lenslet array and travels to a digital image sensor; and (ii)the microscope has an overall magnification that is substantiallygreater than the nominal magnification of the objective lens. In somecases, the microscope does not include any magnifying optical elementother than the tube lens and the objective lens. In some cases: (a) themicroscope has an information budget that is greater by a factor of K²than a modified information budget; (b) K is a positive real number; and(c) the modified information budget is an information budget that themicroscope would have if the microscope were modified by reducing thefocal length of the objective lens and the focal length and diameter ofthe tube lens by a factor of K, without changing the numerical apertureof the microscope and without changing the overall magnification of themicroscope. In some cases: (a) the set of lenslet arrays includes afirst lenslet array and a second lenslet array; (b) the first lensletarray is at first optical distance from the objective lens and thesecond lenslet array is at a second optical distance from the objectivelens; and (c) the first optical distance is not equal to the secondoptical distance. In some cases: (a) the set of lenslet arrays comprisesmultiple lenslet arrays; and (b) the multiple lenslet arrays are allpositioned at the same optical distance from the objective lens. In somecases: (a) each respective lenslet array is shifted by a distancerelative to a specific image sensor that receives light from therespective lenslet array, which distance is in a direction parallel to asurface of the specific image sensor; (b) the distance is different foreach lenslet array in the set of lenslet arrays; and (c) a computer isconfigured to calculate a combined digital image, by computationallycombining digital images that are captured by the image sensors, suchthat the combined image includes overlap regions, each of which overlapregions records light that passed through lenslets of more than lensletarray. In some cases, lenslets in each lenslet array, respectively, arehexagonally packed. In some cases: (a) the microscope has an informationbudget that is greater than a modified information budget by a factor ofK²; (b) the modified information budget is an information budget thatthe microscope would have if the microscope were modified (i) byreducing the focal length and diameter of the tube lens by a factor of Kand (ii) by increasing the ratio of the nominal magnification of theobjective lens to the numerical aperture of the objective lens by afactor of K, and (c) K is a real number greater than 1. Each of thecases described above in this paragraph is an example of the microscopedescribed in the first sentence of this paragraph, and is also anexample of an embodiment of this invention that may be combined withother embodiments of this invention.

In another aspect, this invention is a microscope comprising: (a) afinite conjugate objective lens; (b) a magnifying relay lens forrefracting light from the objective lens; (c) a set of one or morelenslet arrays; and (d) a set of one or more digital image sensors forcapturing light field images; wherein (i) each respective lenslet arrayis positioned such that light from the objective lens and relay lenspasses through the respective lenslet array and travels to a digitalimage sensor, (ii) the microscope has an overall magnification that issubstantially greater than the nominal magnification of the objectivelens, (iii) the microscope has an information budget that is greaterthan a modified information budget by a factor of K², (iv) the modifiedinformation budget is an information budget that the microscope wouldhave if the microscope were modified (A) by removing the relay lens toreduce the overall magnification of the microscope by a factor of K, and(B) by increasing the ratio of the magnification of the objective lensto the numerical aperture of the objective lens by a factor of K, and(v) K is a real number greater than 1. In some cases: (a) the set oflenslet arrays includes a first lenslet array and a second lensletarray; (b) the first lenslet array is at first optical distance from theobjective lens and the second lenslet array is at a second opticaldistance from the objective; and (c) the first optical distance is notequal to the second optical distance. In some cases: (a) the set oflenslet arrays comprises multiple lenslet arrays; and (b) the multiplelenslet arrays are all positioned at the same optical distance from theobjective lens. In some cases: (a) each respective lenslet array isshifted by a distance relative to a specific image sensor that receiveslight from the respective lenslet array, which distance is in adirection parallel to a surface of the specific image sensor; (b) thedistance is different for each lenslet array in the set of lensletarrays; and (c) a computer is configured to calculate a combined digitalimage, by computationally combining digital images that are captured bythe image sensors, such that the combined image includes overlapregions, each of which overlap regions records light that passed throughlenslets of more than lenslet array. In some cases, the lenslets in eachlenslet array, respectively, are hexagonally packed. Each of the casesdescribed above in this paragraph is an example of the microscopedescribed in the first sentence of this paragraph, and is also anexample of an embodiment of this invention that may be combined withother embodiments of this invention.

The above description (including without limitation any attacheddrawings and figures) describes illustrative implementations of theinvention. However, the invention may be implemented in other ways. Themethods and apparatus which are described above are merely illustrativeapplications of the principles of the invention. Other arrangements,methods, modifications, and substitutions by one of ordinary skill inthe art are therefore also within the scope of the present invention.Numerous modifications may be made by those skilled in the art withoutdeparting from the scope of the invention. Also, this invention includeswithout limitation each combination and permutation of one or more ofthe abovementioned implementations, embodiments and features.

What is claimed is:
 1. A microscope comprising: (a) aninfinity-corrected objective lens; (b) a tube lens that is configured torefract light from the objective lens, which tube lens has a focallength that is longer than 200 millimeters; (c) a set of multiplelenslet arrays; and (d) a set of multiple digital image sensors, whichimage sensors are configured to capture light field images; wherein (i)each specific lenslet array in the set of lenslet arrays is positionedin such a way that light from the tube lens passes through the specificlenslet array and travels to an image sensor in the set of imagesensors, and (ii) each lenslet array in the set of lenslet arrays is ata different optical distance from the objective lens than is each otherlenslet array in the set of lenslet arrays.
 2. The microscope of claim1, wherein the microscope includes a set of fiber optic tapers that areconfigured to guide light from the lenslet arrays to the image sensors,in such a way that each fiber optic taper in the set of fiber optictapers guides light from only one of the lenslet arrays to only one ofthe image sensors.
 3. The microscope of claim 1, wherein the microscopeincludes a set of prisms that are configured to guide light from thelenslet arrays to the image sensors, in such a way that each prism inthe set of prisms guides light from only one of the lenslet arrays toonly one of the image sensors.
 4. The microscope of claim 1, wherein:(a) the microscope further includes a microscope tube; and (b) themicroscope is configured in such a way that the image sensors remainstationary relative to the microscope tube.
 5. The microscope of claim1, wherein the microscope is configured in such a way that light in animage circle of the microscope is captured simultaneously by the imagesensors.
 6. The microscope of claim 1, wherein: (a) each specificlenslet array in the set of lenslet arrays is configured to form aspecific image on an image sensor in the set of image sensors, whichspecific image comprises imagelets; (b) the set of lenslet arrays areconfigured in such a way that each imagelet formed by the set of lensletarrays has a diameter d; and (c) the microscope further includes acomputer that is programmed to computationally combine the light fieldimages into a combined digital image, in such a way that, in thecombined image, imagelets overlap to form an overlapped pattern in which(i) the shortest distance between a center of one imagelet and a centerof another imagelet is equal to ${d\;\frac{\sqrt{2}}{2}},$ and (ii) thelongest distance between a center of one imagelet and a center ofanother imagelet is equal to d.
 7. The microscope of claim 1, wherein:(a) each specific lenslet array in the set of lenslet arrays isconfigured to form a specific image on an image sensor in the set ofimage sensors, which specific image comprises imagelets; (b) the set oflenslet arrays are configured in such a way that each imagelet formed bythe set of lenslet arrays has a diameter d; and (c) the microscopefurther includes a computer that is programmed to computationallycombine the light field images into a combined digital image, in such away that, in the combined image, imagelets overlap to form an overlappedpattern in which (i) the shortest distance between a center of oneimagelet and a center of another imagelet is equal to${d\;\frac{\sqrt{3}}{3}},$ and (ii) the longest distance between acenter of one imagelet and a center of another imagelet is equal to d.8. The microscope of claim 1, wherein: (a) each specific lenslet arrayin the set of lenslet arrays is configured to form a specific image onan image sensor in the set of image sensors, which specific imagecomprises imagelets; (b) the set of lenslet arrays are configured insuch a way that each imagelet formed by the set of lenslet arrays has aradius r; and (c) the microscope further includes a computer that isprogrammed to computationally combine the light field images into acombined digital image, in such a way that, in the combined image,imagelets overlap to form an overlapped pattern in which (i) theshortest distance between a center of one imagelet and a center ofanother imagelet is equal to r, and (ii) the longest distance between acenter of one imagelet and a center of another imagelet is equal to r.9. The microscope of claim 1, wherein: (a) each specific lenslet arrayin the set of lenslet arrays is configured to form a specific image onan image sensor in the set of image sensors, which specific imagecomprises imagelets; (b) the set of lenslet arrays are configured insuch a way that each imagelet formed by the set of lenslet arrays has aradius r; and (c) the microscope further includes a computer that isprogrammed to computationally combine the light field images into acombined digital image, in such a way that, in the combined image,imagelets overlap to form an overlapped pattern in which (i) theshortest distance between a center of one imagelet and a center ofanother imagelet is equal to r, and (ii) the longest distance between acenter of one imagelet and a center of another imagelet is equal tor√{square root over (2)}.
 10. The microscope of claim 1, wherein: (a)each specific lenslet array in the set of lenslet arrays is configuredto form a specific image on an image sensor in the set of image sensors,which specific image comprises imagelets; (b) the set of lenslet arraysare configured in such a way that each imagelet formed by the set oflenslet arrays has a radius r; and (c) the microscope further includes acomputer that is programmed to computationally combine the light fieldimages into a combined digital image, in such a way that, in thecombined image, imagelets overlap to form an overlapped pattern in which(i) the shortest distance between a center of one imagelet and a centerof another imagelet is equal to r, and (ii) the longest distance betweena center of one imagelet and a center of another imagelet is equal to$r\;{\frac{\sqrt{2}}{2}.}$
 11. The microscope of claim 1, wherein: (a)each specific lenslet array in the set of lenslet arrays is configuredto form a specific image on an image sensor in the set of image sensors,which specific image comprises imagelets; (b) the set of lenslet arraysare configured in such a way that each imagelet formed by the set oflenslet arrays has a radius r; and (c) the microscope further includes acomputer that is programmed to computationally combine the light fieldimages into a combined digital image, in such a way that, in thecombined image, imagelets overlap to form an overlapped pattern in which(i) the shortest distance between a center of one imagelet and a centerof another imagelet is equal to r, and (ii) the longest distance betweena center of one imagelet and a center of another imagelet is equal to$r\;{\frac{\sqrt{3}}{3}.}$
 12. A microscope comprising: (a) aninfinity-corrected objective lens; (b) a tube lens that are configuredto refract light from the objective lens, which tube lens has a focallength that is longer than 200 millimeters; (c) a set of one or morelenslet arrays that are each configured to refract light from the tubelens; and (d) an image sensor that is configured to capture a lightfield image; wherein the microscope is configured to actuate motion ofthe image sensor in such a way that the image sensor scans the lightfield image.
 13. A microscope comprising: (a) an infinity-correctedobjective lens; (b) a tube lens that is configured to refract light fromthe objective lens, which tube lens has a focal length that is longerthan 200 millimeters; (c) a set of multiple lenslet arrays; and (d) aset of multiple digital image sensors, which image sensors areconfigured to capture light field images; wherein (i) each specificlenslet array in the set of lenslet arrays is positioned in such a waythat light from the tube lens passes through the specific lenslet arrayand travels to an image sensor in the set of image sensors, and (ii)each specific lenslet array in the set of lenslet arrays is at anoptical distance from the objective lens, which optical distance isequal for all of the lenslet arrays in the set of lenslet arrays. 14.The microscope of claim 13, wherein the microscope includes a set offiber optic tapers that are configured to guide light from the lensletarrays to the image sensors, in such a way that each fiber optic taperin the set of fiber optic tapers guides light from only one of thelenslet arrays to only one of the image sensors.
 15. The microscope ofclaim 13, wherein the microscope includes a set of prisms that areconfigured to guide light from the lenslet arrays to the image sensors,in such a way that each prism in the set of prisms guides light fromonly one of the lenslet arrays to only one of the image sensors.
 16. Themicroscope of claim 13, wherein: (a) each specific lenslet array in theset of lenslet arrays is configured to form a specific image on an imagesensor in the set of image sensors, which specific image comprisesimagelets; (b) the set of lenslet arrays are configured in such a waythat each imagelet formed by the set of lenslet arrays has a diameter d;and (c) the microscope further includes a computer that is programmed tocomputationally combine the light field images into a combined digitalimage, in such a way that, in the combined image, imagelets overlap toform an overlapped pattern in which (i) the shortest distance between acenter of one imagelet and a center of another imagelet is equal to${d\;\frac{\sqrt{3}}{3}},$ and (ii) the longest distance between acenter of one imagelet and a center of another imagelet is equal to d.17. The microscope of claim 13, wherein: (a) each specific lenslet arrayin the set of lenslet arrays is configured to form a specific image onan image sensor in the set of image sensors, which specific imagecomprises imagelets; (b) the set of lenslet arrays are configured insuch a way that each imagelet formed by the set of lenslet arrays has aradius r; and (c) the microscope further includes a computer that isprogrammed to computationally combine the light field images into acombined digital image, in such a way that, in the combined image,imagelets overlap to form an overlapped pattern in which (i) theshortest distance between a center of one imagelet and a center ofanother imagelet is equal to r, and (ii) the longest distance between acenter of one imagelet and a center of another imagelet is equal to r.18. The microscope of claim 13, wherein: (a) each specific lenslet arrayin the set of lenslet arrays is configured to form a specific image onan image sensor in the set of image sensors, which specific imagecomprises imagelets; (b) the set of lenslet arrays are configured insuch a way that each imagelet formed by the set of lenslet arrays has aradius r; and (c) the microscope further includes a computer that isprogrammed to computationally combine the light field images into acombined digital image, in such a way that, in the combined image,imagelets overlap to form an overlapped pattern in which (i) theshortest distance between a center of one imagelet and a center ofanother imagelet is equal to r, and (ii) the longest distance between acenter of one imagelet and a center of another imagelet is equal tor√{square root over (2)}.
 19. The microscope of claim 13, wherein: (a)each specific lenslet array in the set of lenslet arrays is configuredto form a specific image on an image sensor in the set of image sensors,which specific image comprises imagelets; (b) the set of lenslet arraysare configured in such a way that each imagelet formed by the set oflenslet arrays has a radius r; and (c) the microscope further includes acomputer that is programmed to computationally combine the light fieldimages into a combined digital image, in such a way that, in thecombined image, imagelets overlap to form an overlapped pattern in which(i) the shortest distance between a center of one imagelet and a centerof another imagelet is equal to r, and (ii) the longest distance betweena center of one imagelet and a center of another imagelet is equal to$r\;{\frac{\sqrt{2}}{2}.}$
 20. The microscope of claim 13, wherein: (a)each specific lenslet array in the set of lenslet arrays is configuredto form a specific image on an image sensor in the set of image sensors,which specific image comprises imagelets; (b) the set of lenslet arraysare configured in such a way that each imagelet formed by the set oflenslet arrays has a radius r; and (c) the microscope further includes acomputer that is programmed to computationally combine the light fieldimages into a combined digital image, in such a way that, in thecombined image, imagelets overlap to form an overlapped pattern in which(i) the shortest distance between a center of one imagelet and a centerof another imagelet is equal to r, and (ii) the longest distance betweena center of one imagelet and a center of another imagelet is equal to$r\;{\frac{\sqrt{3}}{3}.}$